For big software vendors, the quantity of test cases or test programs is usually huge, and will accumulate following every update release of the product. For example, IBM Content Manager has more than 2000 JAVA function test cases developed successively during over 6 years. For guaranteeing the testing quality and designing new test cases, it is very important to learn the coverage of these test cases, that is, which functions of the tested software program have been called by the test cases. However, such a coverage analysis is hard work because of the huge quantity of test cases.
At present, there are several tools which can be used for coverage analysis, but each of them has their own weakness.
Existing heap analyzer tools mainly focus on analyzing memory usage, instead of coverage analysis. Although it is possible to obtain functional coverage information from such tools, they do not report such information directly. Also using such tools can only analyze one heapdump at a time, and cannot analyze multiple heapdumps created by multiple test cases at the same time.
Existing testing analysis tools have special functions for coverage analysis, however, such functions are normally realized by instrumentation, which needs to obtain and modify the source code of the program under test and requires the program under test to execute under the tool's own execution environment, so as to bring about extra costs of purchase, installation, studying and using. Besides, such tools can also only analyze one test case at a time, and cannot perform coverage analysis on the execution results of multiple test cases collectively after executing these test cases.
Apparently, in the field there is a need for an effective coverage analysis solution which can perform coverage analysis on the execution results of multiple test cases at the same time, without requiring the test cases to run in a particular tool's execution environment.